Papers with model architectures
UKP-SQUARE: An Online Platform for Question Answering Research (2022.acl-demo)
Copied to clipboard
Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Gregor Geigle, Max Eichler, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Şahin, Iryna Gurevych
| Challenge: | Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats and require different model architectures and setups. |
| Approach: | They propose an extensible online QA platform that allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
| Outcome: | The proposed tool allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
Tutorial: End-to-End Speech Translation (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Speech translation is the translation of speech in one language typically to text in another, traditionally accomplished through a combination of automatic speech recognition and machine translation. |
| Approach: | This tutorial introduces the techniques used in cutting-edge research on speech translation. |
| Outcome: | The proposed models achieve state-of-the-art performance with end-to-end speech translation for both high- and low-resource languages. |
Retrieval-based Language Models and Applications (2023.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will provide a comprehensive overview of retrieval-based language models. |
| Approach: | This tutorial will provide a comprehensive overview of recent advances in retrieval-based language models. |
| Outcome: | This tutorial will provide a comprehensive overview of recent advances in retrieval-based language models. |
Beyond Sample-Level Feedback: Using Reference-Level Feedback to Guide Data Synthesis (2026.eacl-long)
Copied to clipboard
| Challenge: | High-quality instruction-tuning data is crucial for Large Language Models (LLMs) but it imposes a quality ceiling where models trained on the data cannot outperform the LLM generating it. |
| Approach: | They propose a paradigm that extracts desirable characteristics from carefully curated reference samples to guide the synthesis of higher-quality instruction-response pairs. |
| Outcome: | The proposed paradigm outperforms traditional sample-level feedback methods and generalizes across model architectures. |
Neural Retriever and Go Beyond: A Thesis Proposal (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing neural retrievers are developed for pure-text queries, which prevents them from handling multi-modality queries. |
| Approach: | They propose methods to address issues of existing neural retrievers from three angles . they propose new model architectures, IR-oriented pretraining tasks and generating large scale training data . |
| Outcome: | The proposed methods address the abovementioned issues of neural retrievers from three angles and generate large scale training data. |
Template-guided Grammatical Error Feedback Comment Generation (2023.eacl-srw)
Copied to clipboard
| Challenge: | Writing corrective feedback on learner text is widespread in language education, but it can be time-consuming for teachers. |
| Approach: | They propose to use feedback comment generation to generate explanatory notes for learners by categorizing comments and constraining outputs of noisy classes. |
| Outcome: | The proposed scheme can be used to generate feedback comment corpora using a broader scope than existing typologies focused on error correction. |
Establishing Trustworthiness: Rethinking Tasks and Model Evaluation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Language understanding is a multi-faceted cognitive capability, which the Natural Language Processing community has striven to model computationally for decades. |
| Approach: | They propose to rethink what constitutes tasks and model evaluation in NLP and pursue a more holistic view on language, placing trustworthiness at the center. |
| Outcome: | The proposed models are based on generative models and are being deployed in more real-world scenarios, including previously unforeseen zero-shot setups. |
PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications (2025.emnlp-industry)
Copied to clipboard
Hitesh Laxmichand Patel, Amit Agarwal, Srikant Panda, Hansa Meghwani, Karan Dua, Paul Li, Tao Sheng, Sujith Ravi, Dan Roth
| Challenge: | Existing evaluation metrics for Multimodal Large Language Models (MLLMs) are inadequate to assess their robustness to irrelevant or distracting visual context. |
| Approach: | They propose a patch-context-robustness index to measure MLLMs' robustness to visual context variations. |
| Outcome: | The proposed score measures the robustness of MLLMs to visual contexts across 15 vision-language benchmarks. |
Normalized Contrastive Learning for Text-Video Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Cross-modal contrastive learning suffers from incorrect normalization of the sum retrieval probabilities of each text or video instance. |
| Approach: | They propose a normalized contrastive learning algorithm that normalizes the sum retrieval probabilities of each instance so that every text and video instance is fairly represented. |
| Outcome: | Empirical results show that NCL brings significant gains in text-video retrieval on different model architectures without any architecture engineering. |
ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models (2022.tacl-1)
Copied to clipboard
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
| Challenge: | a number of pre-trained language models use sequences of tokens corresponding to word units . token-free models that operate directly on raw text have many advantages . |
| Approach: | They propose a standard Transformer architecture that can be used to process byte sequences . they also characterize trade-offs in terms of parameter count, training FLOPs, and inference speed . |
| Outcome: | The proposed model is more robust to noise and more robust on spelling and pronunciation tasks. |
Friend-training: Learning from Models of Different but Related Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Current self-training methods focus on improving model performance on a single task. |
| Approach: | They propose a cross-task self-training framework where models trained to do different tasks are used in iterative training, pseudo-labeling, and retraining processes to help each other for better selection of pseudo-labeled labels. |
| Outcome: | The proposed framework achieves the best performance compared to baselines on two dialogue understanding tasks. |
LibKGE - A knowledge graph embedding library for reproducible research (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Knowledge graph embedding models are trained to predict false triples and high scores for true triples. |
| Approach: | LibKGE is an open-source PyTorch-based library for training, hyperparameter optimization, and evaluation of knowledge graph embedding models for link prediction. |
| Outcome: | LibKGE provides implementations of common knowledge graph embedding models and training methods, and new ones can be easily added. |
The Shape of Vulnerability: How Adversarial Perturbations Reshape the Topology of Language Model Latent Spaces (2026.acl-srw)
Copied to clipboard
Angelina Tsai, Shreya Subramanian, Catherine Liu, Kimberly Lopez, Leif Zinn-Brooks, Alexia E. Schulz, Adaku Uchendu
| Challenge: | Large Language Models (LLMs) have unprecedented capabilities, but they pose security concerns . current adversarial attacks exploit vulnerabilities in the embedding space of language models, allowing attackers to bypass safety guardrails and cause significant harmful consequences. |
| Approach: | They propose to use topological data analysis to characterize how adversarial perturbations act on text inputs by computing persistent homology metrics from attention maps across different model architectures. |
| Outcome: | The proposed visualizations show that adversarial perturbations alter higher-dimensional topological features in ways that distinguish them from clean, non-adversarial inputs. |
Discourse-Based Sentence Splitting (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Sentence splitting is a key component of sentence simplification and has been shown to help human comprehension. |
| Approach: | They propose to use a discourse connective to generate a sentence that is shorter than the input text. |
| Outcome: | The proposed models outperform end-to-end models in learning the various ways of expressing a discourse relation but generate text that is less grammatical. |
Reverse Engineering Configurations of Neural Text Generation Models (2020.acl-main)
Copied to clipboard
| Challenge: | Recent advances in neural text generation modeling have raised concerns about how such approaches might be used in malicious ways. |
| Approach: | They propose to distinguish which of several variants of a given model generated some piece of text by performing diagnostic tests. |
| Outcome: | The proposed method identifies which of several variants of a given model generated some piece of text and if so, if it is more sensitive to different modeling choices than previously thought. |
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)
Copied to clipboard
Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing
| Challenge: | Texar is an open-source text generation toolkit that supports a broad set of text generation tasks. |
| Approach: | They introduce Texar, an open-source text generation toolkit that supports text generation tasks. |
| Outcome: | Texar supports machine translation, summarization, dialog, content manipulation, and more. |
Cross-Task Generalization Abilities of Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | a thesis proposal advocates for the crucial role of cross-task generalization in NLP systems. |
| Approach: | They propose to benchmark cross-task generalization abilities with diverse NLP tasks . they also propose to develop model architectures for improving cross- task generalization . |
| Outcome: | This paper compares cross-task generalization abilities with diverse NLP tasks . it also analyzes and predicts the generalization landscape of current state-of-the-art large language models . |
Variable Typing: Assigning Meaning to Variables in Mathematical Text (N18-1)
Copied to clipboard
| Challenge: | Scientific documents rely on mathematics to communicate ideas and results . textual contexts are strong domainspecific conventions governing how content is presented . |
| Approach: | They introduce a task of assigning one mathematical type to each variable in a sentence . they also introduce 'variable typing' task that focuses on assignment of meaning to variables . |
| Outcome: | The proposed model is the best performing model on an extrinsic task, the authors show . their model is compared to a formula index only containing raw symbols . |
Discovering Better Model Architectures for Medical Query Understanding (2021.naacl-industry)
Copied to clipboard
| Challenge: | Neural architecture search (NAS) has attracted intense attention in computer vision and NLP. |
| Approach: | They propose to use neural architecture search to optimize model architectures for medical questions . they propose to modify the ENAS method to accelerate and stabilize the search results . |
| Outcome: | The proposed approach outperforms baseline models on two medical questions . it is compared with other NAS methods and shows that it provides the best results . |
GeoIndia: A Seq2Seq Geocoding Approach for Indian Addresses (2024.emnlp-industry)
Copied to clipboard
| Challenge: | a new geocoding system for Indian addresses addresses is needed for logistics, urban planning and location-based services. |
| Approach: | They propose a geocoding system for Indian addresses using hierarchical H3-cell prediction using a Seq2Seq framework. |
| Outcome: | The proposed system outperforms existing geocoding platforms in accuracy and reliability across multiple Indian states. |
Abstractive Summarizers are Excellent Extractive Summarizers (2023.acl-short)
Copied to clipboard
| Challenge: | Abstractive summarization systems have traditionally been fragmented, limiting the benefits of compatible models. |
| Approach: | They propose three new inference algorithms using sequence-to-sequence architectures to model extractive summarization with an abstractive summmarization system. |
| Outcome: | The proposed algorithms outperform existing models on CNN and Dailymail and show that they are more efficient than existing models. |
Augur: Modeling Covariate Causal Associations in Time Series via Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have emerged as a promising avenue for time series forecasting . existing approaches face limitations such as marginalized role in model architectures and lack of interpretability. |
| Approach: | They propose a framework that exploits LLM causal reasoning to discover and use directed causal associations among covariates. |
| Outcome: | The proposed model improves predictive accuracy while yielding transparent, traceable reasoning about variable interactions. |
LexSym: Compositionality as Lexical Symmetry (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generalize compositional models fail to generalise from small datasets. |
| Approach: | They propose a domain-general and model-agnostic formulation of compositionality as a constraint on symmetries of data distributions rather than models. |
| Outcome: | The proposed procedure matches or surpasses state-of-the-art, task-specific models on COGS semantic parsing, SCAN and Alchemy instruction following, and CLEVR-CoGenT visual question answering datasets. |
Text-to-Text Automatic Story Generation: A Survey (2026.eacl-srw)
Copied to clipboard
| Challenge: | Automated story generation aims to produce coherent, engaging, and contextually consistent narratives with minimal or no human involvement . despite advances in large language models, maintaining narrative coherence, character consistency, storyline diversity, and plot controllability in generating stories is still challenging. |
| Approach: | They propose to develop new evaluation metrics and better data sets to support automatic story generation. |
| Outcome: | The proposed evaluation metrics and better datasets will improve narrative coherence and consistency and explore practical applications of story generation. |
Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-speech (TTS) has advanced from generating natural-sounding speech to enabling fine-grained control over speech attributes. |
| Approach: | They provide a review of controllable TTS methods from traditional control techniques to emerging approaches using natural language prompts. |
| Outcome: | The proposed methods are based on models, strategies, and features, and summarize challenges, datasets, and evaluations. |
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection (2025.acl-long)
Copied to clipboard
| Challenge: | a rapid expansion of memes on social media highlights the need for effective methods to detect harmful content. |
| Approach: | They propose a multi-agent framework for zero-shot harmful meme detection that does not rely on annotated data. |
| Outcome: | The proposed framework outperforms existing zero-shot approaches on three meme datasets. |
Joint Universal Syntactic and Semantic Parsing (2021.tacl-1)
Copied to clipboard
| Challenge: | Several attempts have been made to jointly parse syntax and semantics, but this trade-off is not well understood. |
| Approach: | They propose multiple model architectures that exploit the rich syntactic and semantic annotations contained in the Universal Decompositional Semantics dataset to obtain state-of-the-art results. |
| Outcome: | The proposed model outperforms existing models in 8 languages and their results are consistent across languages. |
Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations (2022.aacl-main)
Copied to clipboard
Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, Alison Q. O’Neil
| Challenge: | Existing approaches to generate text radiology reports are prone to errors and poor clinical accuracy. |
| Approach: | They propose a two-step pipeline that subdivides the problem into factual triple extraction followed by free-text report generation. |
| Outcome: | The proposed pipeline shows that the generated reports exhibit realistic style but lack clinical accuracy. |
The Illusion of Insight in Reasoning Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evidence suggests that reasoning models exhibit "Aha!" moments . however, it remains unclear whether such intrinsic shifts in reasoning strategy improve performance . |
| Approach: | They analyze reasoning traces, training checks, three reasoning domains to detect mid-reasoning shifts . they find that reasoning shifts are rare, do not become more frequent with training . |
| Outcome: | The proposed method detects mid-reasoning shifts and instrument training runs. |
Improving Cross-lingual Transfer through Subtree-aware Word Reordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multilingual language models are not effective when dealing with less-represented languages. |
| Approach: | They propose a powerful reordering method that learns word-order patterns conditioned on the syntactic context from a small amount of annotated data. |
| Outcome: | The proposed method outperforms baselines on a variety of tasks and is effective in both zero-shot and few-shot scenarios. |
Cross-lingual Few-Shot Learning on Unseen Languages (2022.aacl-main)
Copied to clipboard
| Challenge: | Large pre-trained language models have demonstrated the ability to obtain good performance on downstream tasks with limited examples in resource-rich languages. |
| Approach: | They propose to use a downstream sentiment analysis task to analyze the effectiveness of several few-shot learning strategies across 12 languages, including 8 unseen languages, to compare results. |
| Outcome: | The proposed model, XLM-R, gives the best performance on a task with few examples in resource-rich languages. |
WALNUT: A Benchmark on Semi-weakly Supervised Learning for Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on weak supervision for NLU focus on a specific task or simulate weak supervision signals from ground-truth labels. |
| Approach: | They propose a benchmark to advocate and facilitate research on weak supervision for NLU . they use document-level and token-level prediction tasks as examples . |
| Outcome: | The proposed benchmark advocates and facilitates research on weak supervision for NLU tasks. |
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)
Copied to clipboard
Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations. |
| Approach: | They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference. |
| Outcome: | The proposed method significantly mitigates object hallucinations across various model architectures. |
Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to enhance dense retrieval models are unwieldy, such as requiring explicit supervision, complex model architectures, or massive external models. |
| Approach: | They propose an unsupervised method to enhance passage retrieval in zero-shot settings by iterating a loop that a dense retriever learns from supervision signals provided by a reranker. |
| Outcome: | The proposed method outperforms leading supervised and unsupervised retrievers on the BEIR benchmark while showing strong adaptation abilities to tasks and domains that were unseen during training. |
Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings (2023.emnlp-main)
Copied to clipboard
| Challenge: | XLMs can support cross-lingual transfer learning with little to no additional training data. |
| Approach: | They describe a mechanism for cross-lingual transfer learning by measuring the properties of the initial token embedding layer. |
| Outcome: | The proposed model can be used to support cross-lingual transfer learning . the initial token embedding layer is expressive and interpretable . |
Consistency Regularization Training for Compositional Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing neural models have difficulty generalizing to unseen combinations of seen components. |
| Approach: | They propose to improve the capability of Transformer on compositional generalization by consistency regularization training without modifying model architectures. |
| Outcome: | The proposed model performs well on semantic parsing and machine translation benchmarks. |
Towards Understanding Neural Machine Translation with Word Importance (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has advanced the state-of-the-art on various language pairs, but the interpretability of NMT remains unsatisfactory. |
| Approach: | They propose to attribute NMT output to every input word using a gradient-based method to measure word importance. |
| Outcome: | The proposed method is superior on identifying input words with higher influence on translation performance. |
ConTempo: A Unified Temporally Contrastive Framework for Temporal Relation Extraction (2024.findings-acl)
Copied to clipboard
| Challenge: | Temporal relation extraction (TRE) is a task of classifying temporal relations between events conveyed in narratives. |
| Approach: | They propose a Temporally Contrastive learning model that increases the model’s awareness of the meaning of temporal relations by leveraging their symmetric or antisymmetric properties. |
| Outcome: | The proposed model improves the model's representation of meaning of temporal relations and its ability to integrate with the underlying temporal calculus. |
Systematic Generalization in Language Models Scales with Information Entropy (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for assessing compositional behavior are unclear on how to measure the difficulty of a systematic generalization problem. |
| Approach: | They propose a framework for measuring entropy in a sequence-to-sequence task and a method for measuring it. |
| Outcome: | The proposed framework scales with the entropy of the distribution of component parts in the training data. |
Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decoding (2023.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art entity tracking approaches either design complicated model architectures or rely on task-specific pre-training to achieve good results. |
| Approach: | They propose a multi-task learning-enabled entity tracking approach that utilizes knowledge gained from general domain tasks to improve entity tracking. |
| Outcome: | The proposed approach achieves state-of-the-art on two popular entity tracking datasets, even though it does not require any task-specific architecture design or pre-training. |
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment (2023.emnlp-main)
Copied to clipboard
| Challenge: | Inference is estimated to make up 80 to 90% of ML cloud computing demand . |
| Approach: | They propose to identify bottlenecks in deep learning frameworks that are causing the disparity in model latency as hardware speed increases over time. |
| Outcome: | The proposed models show that the framework tax is increasing as the hardware speed increases over time. |
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen success in the use of deep neural networks on text summarization, but there is no clear understanding of why they perform so well or how they might be improved. |
| Approach: | They propose to use different types of model architectures to improve extractive summarization systems. |
| Outcome: | The proposed framework achieves state-of-the-art on CNN/DailyMail by a large margin based on observations and analysis. |
From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has recently advanced the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose a method that incorporates partial solution prefixes from expert demonstrations to guide the policy. |
| Outcome: | The proposed methods outperform strong baselines, yielding faster convergence and a higher performance ceiling. |
Detection of Adversarial Prompts with Model Predictive Entropy (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly deployed in high-impact scenarios raising concerns about their safety and security. |
| Approach: | They propose an attack-agnostic pipeline for detecting adversarial inputs without prior knowledge of attack specifications. |
| Outcome: | The proposed pipeline outperforms traditional defenses in terms of adaptability and resource efficiency. |
An Empirical Study of Multimodal Model Merging (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that model merging can generate a multi-task solution without synchronous training. |
| Approach: | They propose to merge vision, language, and cross-modal transformers of a modality-specific architecture to create a parameter-efficient architecture. |
| Outcome: | The proposed model merging outperforms naive models on various tasks with improvements of 3% on VQA, 7% on COCO retrieval, 25% on NLVR2, 14% on Flickr30k and 3% ADE20k. |
Longtonotes: OntoNotes with Longer Coreference Chains (2023.findings-eacl)
Copied to clipboard
Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, Mrinmaya Sachan
| Challenge: | Using Ontonotes, documents in certain genres were split into smaller parts for ease of annotation. |
| Approach: | They propose to merge annotations from documents split into smaller parts in Ontonotes for ease of annotation. |
| Outcome: | The proposed corpus restores documents to their original form, revealing dramatic increases in length in certain genres. |
Large Language Models with Controllable Working Memory (2023.findings-acl)
Copied to clipboard
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, Sanjiv Kumar
| Challenge: | Large language models (LLMs) have led to a series of breakthroughs in natural language processing due to the massive amounts of world knowledge they memorize during pretraining. |
| Approach: | They propose a method to inject counterfactual and irrelevant contexts into standard supervised datasets to strengthen both controllability and robustness. |
| Outcome: | The proposed method improves controllability and robustness across model architectures and sizes. |
AntCritic: Argument Mining for Free-Form and Visually-Rich Financial Comments (2024.lrec-main)
Copied to clipboard
| Challenge: | Argument mining is a thriving task in natural language processing, but its generalization is limited by existing datasets. |
| Approach: | They propose to use a dataset to help model argument mining . the dataset AntCritic supports both argument component detection and argument relation prediction tasks. |
| Outcome: | The proposed model can detect arguments and identify their relationships automatically. |
Jigsaw Pieces of Meaning: Modeling Discourse Coherence with Informed Negative Sample Synthesis (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on creating “informed” incoherent samples for coherence modeling has focused on permutations of a coherent document . |
| Approach: | They propose to use Constituency trees, Part-of-speech, semantic overlap to create “informed” negative samples that better represent or mimic incoherence. |
| Outcome: | The proposed methods improve the quality of the negative sample. |
SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation (N18-1)
Copied to clipboard
| Challenge: | Experimentally, we find that the proposed models consistently outperform models that encapsulate single-style or average-style language generation capabilities. |
| Approach: | They propose a family of model architectures capable of capturing both generic language characteristics via shared model parameters, as well as particular style characteristics via private model parameters. |
| Outcome: | The proposed models outperform models that encapsulate single-style or average-style language generation capabilities. |
Trained on 100 million words and still in shape: BERT meets British National Corpus (2023.findings-eacl)
Copied to clipboard
| Challenge: | masked language models are trained on ever larger corpora, but pre-training on a modestly-sized but representative, well-balanced, and publicly available corpus can reach better performance than the original BERT model. |
| Approach: | They propose an optimized LM architecture called LTG-BERT that can be used to train a competitive language model on a small and standardizable corpus. |
| Outcome: | The proposed architecture outperforms the original English BERT model on a representative, well-balanced and publicly available corpus. |
When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain (2022.emnlp-main)
Copied to clipboard
Raj Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, Diyi Yang
| Challenge: | Pre-trained language models have shown impressive performance on a variety of tasks and domains. |
| Approach: | They propose a domain specific financial LANGuage model which uses financial keywords and phrases for better masking. |
| Outcome: | The proposed model outperforms existing models on a variety of tasks and domains. |
Improving Chain-of-Thought for Logical Reasoning via Attention-Aware Intervention (2026.findings-eacl)
Copied to clipboard
| Challenge: | Modern logical reasoning with LLMs relies on employing complex interactive frameworks that decompose the reasoning process into subtasks solved through carefully designed prompts or requiring external components, which limit their scalability. |
| Approach: | They propose a non-interactive, end-to-end framework for reasoning tasks that enables reasoning to emerge within the model itself. |
| Outcome: | The proposed framework improves generalization while preserving analyzability without external resources. |
FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression (2026.findings-eacl)
Copied to clipboard
| Challenge: | Low-rank decomposition methods suffer from accuracy degradation and expensive calibration procedures. |
| Approach: | They propose a fast and accurate, training-free structural compression method based on fine-grained low-rank transformations in the activation space. |
| Outcome: | The proposed method outperforms pruning baselines in generalization and downstream performance while delivering inference speedups. |
DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in video-text retrieval models have limited training data annotations. |
| Approach: | They propose a Video-Text Retrieval Paradigm with Relevance-based Augmentation which enhances video and text data using large foundation models to learn more generalized features. |
| Outcome: | The proposed method improves video-text retrieval performance over existing methods. |
Cultural Learning-Based Culture Adaptation of Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for adapting large language models to diverse cultural values often rely on prompt engineering. |
| Approach: | They propose a framework for enhancing LLM alignment with cultural values based on cultural learning that leverages simulated social interactions to generate role-playing scenarios. |
| Outcome: | The proposed framework improves cultural value alignment across various model architectures measured using World Value Survey data. |
GUMsley: Evaluating Entity Salience in Summarization for 12 English Genres (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing work on salient entity extraction relies on crowdsourcing or user statistics to derive labels for entities. |
| Approach: | They propose a dataset that defines salience using human summaries and shows high agreement between annotations based on whether a source entity is mentioned in the summary. |
| Outcome: | The proposed dataset shows that pre-trained models and zero-shot LLM prompting fail to capture salient entities in generated summaries. |
Self-Adaptive Sampling for Accurate Video Question Answering on Image Text Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Image–text models (ITMs) are the prevalent architecture to solve video question–answering tasks, which requires only a few input frames to save huge computational cost compared to video–language models. |
| Approach: | They propose a sampling method based on question–frame correlation that is efficient for the few-frame situations. |
| Outcome: | The proposed method can boost the performance of image–text pretrained models and have a wide application scenario in terms of model architectures and dataset types. |
Knowing More About Questions Can Help: Improving Calibration in Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on calibration focuses on model confidence, such as the max probability of the predicted class. |
| Approach: | They propose a calibration method which estimates whether model correctly predicts answer for each question. |
| Outcome: | The proposed calibration method achieves 5-10% gains on reading comprehension benchmarks. |
Improving Named Entity Recognition by Jointly Learning to Disambiguate Morphological Tags (C18-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) taggers require external morphological disambiguation tools to function which are hard to obtain or non-existent for many languages. |
| Approach: | They propose a model which jointly learns NER and MD taggers in languages for which one can provide a list of candidate morphological analyses. |
| Outcome: | The proposed model can be trained independently of the morphological annotation schemes, and it performs competitively. |
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation (2022.findings-acl)
Copied to clipboard
Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, Xipeng Qiu
| Challenge: | Existing methods to measure instance difficulty use generalization and threshold-tuning . a new approach to learn to exit is based on hash functions to assign tokens to a fixed exiting layer. |
| Approach: | They propose a Hash-based Early Exiting approach that replaces learn-to-exit modules with hash functions to assign each token to a fixed exiting layer. |
| Outcome: | The proposed approach improves on learning to exit and predicting instance difficulty. |
GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models (2025.findings-acl)
Copied to clipboard
Mufan Qiu, Xinyu Hu, Fengwei Zhan, Sukwon Yun, Jie Peng, Ruichen Zhang, Bhavya Kailkhura, Jiekun Yang, Tianlong Chen
| Challenge: | Foundation models for single-cell RNA sequencing ignore biological prior knowledge encoded in gene regulatory relationships and fail to leverage multi-omics signals. |
| Approach: | They propose a framework that integrates multi-scale gene regulatory networks into RNA foundation model training. |
| Outcome: | The proposed framework improves on state-of-the-art models on three downstream tasks . it integrates multi-scale gene regulatory networks (GRNs) from multi-omics data into training . |
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing long-text evaluation benchmarks, such as L-Eval and LongBench, focus on QA and summarization tasks. |
| Approach: | They propose a length-adaptable benchmark for evaluating the long-context understanding of large language models. |
| Outcome: | The proposed benchmarks do not cover ultralong settings (100k+ tokens) and are difficult to evaluate across different length ranges. |
Phone Features Improve Speech Translation (2020.acl-main)
Copied to clipboard
| Challenge: | End-to-end models for speech translation more tightly couple speech recognition (ASR) and machine translation (MT) compared to cascades, but performance gap remains in low-resource conditions . |
| Approach: | They propose two methods to incorporate phone features into current neural speech translation models. |
| Outcome: | The proposed models outperform existing models and cascades by up to 9 BLEU on low-resource conditions. |
Paraphrase Generation and Evaluation on Colloquial-Style Sentences (2020.lrec-1)
Copied to clipboard
| Challenge: | a new study investigates the quality and novelty of generated paraphrases . paraphrase models can be used for information retrieval and data mining . |
| Approach: | They use state-of-the-art neural machine translation models trained on the Opusparcus corpus to generate paraphrases in six languages. |
| Outcome: | The proposed model outperforms the existing model on human evaluation in five of the six languages. |
Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evaluation methods do not provide insight into how well a language model captures distinct linguistic skills essential for language understanding and reasoning. |
| Approach: | They propose a new format of NLI benchmark for evaluation of broad-coverage linguistic phenomena using a set of datasets and an evaluation procedure for diagnosing how well a language model captures reasoning skills. |
| Outcome: | The proposed model can diagnose model behavior and verify model learning quality. |
MDR: Model-Specific Demonstration Retrieval at Inference Time for In-Context Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieval-based in-context learning ignore model biases and fail to retrieve the most appropriate demonstrations for different LLMs. |
| Approach: | They propose a model-specific demonstration retrieval method that considers the biases of different LLMs at inference time. |
| Outcome: | The proposed method improves performance on seen and unseen tasks with multi-scale inference LLMs by up to 41.2%. |
Tokenizer Choice For LLM Training: Negligible or Crucial? (2024.findings-naacl)
Copied to clipboard
Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, Nicolas Flores-Herr
| Challenge: | Recent success of large language models has been driven by curating the training dataset composition, scaling of model architectures and advancements in pretraining objectives, leaving tokenizer influence as a blind spot. |
| Approach: | They conduct a comprehensive study on the influence of tokenizer choice on LLM downstream performance by training 24 mono- and multilingual LLMs at a 2.6B parameter scale. |
| Outcome: | The proposed model can significantly impact the model's downstream performance and training costs. |
SiMFy: A Simple Yet Effective Approach for Temporal Knowledge Graph Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for temporal knowledge graph reasoning suffer from low training efficiency and insufficient generalization ability. |
| Approach: | They propose a temporal knowledge graph reasoning approach that uses multilayer perceptron to model the structural dependencies of events and adopts a fixed-frequency strategy to incorporate historical frequency during inference. |
| Outcome: | The proposed model achieves state-of-the-art performance with faster convergence speed and better generalization ability. |
Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales (2024.findings-naacl)
Copied to clipboard
| Challenge: | Saliency post-hoc explainability methods are important tools for understanding complex NLP models, but they may not align with human intuition, making the explanations not plausible. |
| Approach: | They propose a method for incorporating rationales into text classification models by augmenting the standard cross-entropy loss with a novel loss function inspired by contrastive learning. |
| Outcome: | The proposed approach enhances the plausibility of post-hoc explanations while preserving their faithfulness. |
A Diagnostic Study of Explainability Techniques for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing explainability techniques that can be produced post-hoc with already trained models are lacking a definitive guide on how to choose one given a particular task and model architecture. |
| Approach: | They propose to use a list of diagnostic properties to evaluate existing explainability techniques to compare them with human annotations of salient input regions. |
| Outcome: | The proposed list compares a set of explainability techniques on downstream text classification tasks and neural network architectures. |
Efficient Citer: Tuning Large Language Models for Enhanced Answer Quality and Verification (2024.findings-naacl)
Copied to clipboard
Marzieh Tahaei, Aref Jafari, Ahmad Rashid, David Alfonso-Hermelo, Khalil Bibi, Yimeng Wu, Ali Ghodsi, Boxing Chen, Mehdi Rezagholizadeh
| Challenge: | Existing models with explicit citations lack the ability to verify information generated by these models. |
| Approach: | They construct a citation training dataset and fine-tune two models to address the challenge of explicit citations efficiently. |
| Outcome: | The proposed models surpass ChatGPT and exhibit exceptional out-of-domain generalization in both human and automatic evaluation. |
Fluent Translations from Disfluent Speech in End-to-End Speech Translation (N19-1)
Copied to clipboard
| Challenge: | Disfluency removal is an intermediate step between speech recognition and machine translation (MT) with the rise of end-to-end speech translation systems, disfluency recognition and removal needs to be incorporated into the model architectures or handled as a post-processing step. |
| Approach: | They propose to use a sequence-to-sequence model to translate from noisy, disfluent speech to fluent text with disfluencies removed using the recently collected ‘copy-edited’ references for the Fisher Spanish-English dataset. |
| Outcome: | The proposed model generates fluent translations from disfluent speech using the recently collected ‘copy-edited’ references for the Fisher Spanish-English dataset. |
Domain Adaptation of Neural Machine Translation by Lexicon Induction (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is sensitive to domain shift, resulting in failure for sentences with large numbers of unknown words and lack of supervision for domain-specific words. |
| Approach: | They propose an unsupervised method which fine-tunes a pre-trained out-of-domain NMT model using a pseudo-in-domain corpus. |
| Outcome: | The proposed method improves in five domains without using in-domain parallel sentences and up to 2 BLEU over strong back-translation baselines. |
DoT: An efficient Double Transformer for NLP tasks with tables (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that transformer-based approaches to NLP tasks are slow and require computational and memory costs. |
| Approach: | They propose a transformer-based model that decomposes a problem into two sub-tasks and a pruning transformer that takes as input the pruning scores. |
| Outcome: | The proposed model improves training and inference time by at least 50% for a small drop in accuracy and also enables the model to maintain similar accuracy as slower baseline models. |
A Framework for Bidirectional Decoding: Case Study in Morphological Inflection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing encoder-decoders that generate sequences from left to right are prone to errors due to the "snowballing" effect. |
| Approach: | They propose a transformer-based encoder-decoder framework that produces sequences from the "outside-in" they argue that this approach is more principled than prior bidirectional decoders . |
| Outcome: | The proposed model beats the current system by over 4.7 and 2.7 points in accuracy on 2022 and 2023 tasks. |
Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Personality-aware LLMs exhibit implicit personalities in their generation, but reliably controlling or aligning these traits to meet specific needs remains an open challenge. |
| Approach: | They propose a pipeline that extracts hidden state activations from transformer layers using the Big Five Personality Traits framework. |
| Outcome: | The proposed model extracts hidden state activations from transformer layers using the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism) |
”I Never Said That”: A dataset, taxonomy and baselines on response clarity classification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Equivocation and ambiguity in public speech are well-studied discourse phenomena . a new taxonomy aims to detect and classify response clarity in political interviews . |
| Approach: | They propose a taxonomy that uses Large Language Models and human annotations to detect and classify response clarity in political interviews. |
| Outcome: | The proposed taxonomy combines ChatGPT and human annotations to identify clarity in political questions . it provides a fine-grained taxonomies for evasion techniques related to unclear, ambiguous responses . |
Reducing Gender Bias in Abusive Language Detection (D18-1)
Copied to clipboard
| Challenge: | Abusive language detection models tend to be biased toward identity words of a certain group of people . recent studies have raised concerns about the robustness of such systems . |
| Approach: | They propose to use debiased word embeddings, gender swap data augmentation to reduce model bias . they also propose to fine-tune models with a larger corpus to correct such bias if needed . |
| Outcome: | The proposed methods reduce model bias by 90-98% and can be extended to correct model bias in other scenarios. |
Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to explain neural network models are computationally inefficient for text inputs. |
| Approach: | They propose a method to implicitly detect word correlations by grouping correlated words from input text pairs together and measuring their contribution to corresponding NLP tasks. |
| Outcome: | The proposed method is evaluated with two different model architectures across four datasets. |
TVShowGuess: Character Comprehension in Stories as Speaker Guessing (2022.naacl-main)
Copied to clipboard
| Challenge: | Our proposed task, TVShowGuess, builds on the scripts of TV series and takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and the dialogues. |
| Approach: | They propose a task that takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and the dialogues. |
| Outcome: | The proposed models outperform baselines, yet lag behind human performance. |
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods for text summarization systems are limited to in-domain setting, where supervised pre-trained models are evaluated on the same dataset. |
| Approach: | They propose to use a cross-dataset evaluation approach to evaluate different summarization systems in a multi-domain setting. |
| Outcome: | The proposed model can be used to evaluate text summarization systems on different datasets. |
Analyzing Key Factors Influencing Emotion Prediction Performance of VLLMs in Conversational Contexts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that large language models and vision large language model (VLLMs) possess EI and the ability to understand emotional stimuli in the form of text and images. |
| Approach: | They analyze the key elements affecting the emotion prediction performance of VLLMs in conversational contexts. |
| Outcome: | The proposed model performance was compared with other models in a conversational context. |
Are Pretrained Convolutions Better than Pretrained Transformers? (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research has shown promise in entirely convolutional, or CNN, architectures, but they have not been explored using the pre-train-fine-tune paradigm. |
| Approach: | They propose to use the pre-train-fine-tune paradigm to study convolutional models. |
| Outcome: | The proposed architectures outperform Transformers in certain scenarios, but with caveats. |
Incentivizing Strong Reasoning from Weak Supervision (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive performance on reasoning-intensive tasks, but enhancing their reasoning abilities typically relies on expensive high-quality demonstrations and reinforcement learning. |
| Approach: | They propose to incentivize reasoning abilities of large language models without expensive demonstrations and reinforcement learning. |
| Outcome: | The proposed model can recover 94% of the gains of expensive RL at a fraction of the cost. |
VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Format (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies on video large language models focus on model architectures and training datasets . interaction format between user and model is unsatisfactory for time-sensitive tasks . |
| Approach: | They propose a video-text duet interaction format that allows for continuous playback of the video . when a text message ends, the video continues to play, similar to the alternative of two performers in a duet. |
| Outcome: | The proposed format improves performance on time-sensitive tasks with minimal training efforts. |
Context Analysis for Pre-trained Masked Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models that learn contextualized word representations from a large un-annotated corpus have become a standard component for many downstream NLP tasks. |
| Approach: | They propose to use a masking and gradient approach to evaluate the impact of context on the word representation. |
| Outcome: | The proposed model architectures are architecture agnostic and gradient based. |
Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large vision-language models are often not open-source due to preventing abuse or commercial factors. |
| Approach: | They propose a method for parameter-efficient fine-tuning to improve model accessibility . large models are often not open-source due to preventing abuse or commercial factors . they propose implementing a lightweight adapter over the output feature of an inaccessible model . |
| Outcome: | The proposed methods improve on 11 benchmarks and are made publicly available. |
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that the flatness of the local minimum correlates well with better generalization. |
| Approach: | They propose to use a method encouraging convergence to a flatter minimum to fine-tune PLMs. |
| Outcome: | The proposed method outperforms state-of-the-art methods on NLP tasks without extra computation cost. |
Self-Foveate: Enhancing Diversity and Difficulty of Synthesized Instructions from Unsupervised Text via Multi-Level Foveation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models rely on human effort for data annotation. |
| Approach: | They propose an unsupervised method that generates unsupervised instruction from unsupervised text using a "Micro-Scatter-Macro" method that excavates fine-grained information embedded in unsupervised texts. |
| Outcome: | The proposed method improves diversity and difficulty of synthesized instructions across multiple unsupervised corpora and diverse model architectures. |
What’s under the hood: Investigating Automatic Metrics on Meeting Summarization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation metrics do not capture meeting-specific errors, leading to ineffective assessment. |
| Approach: | They examine the relationship between established metrics and human evaluations to determine what challenges and errors are captured by correlating metric scores with human evaluation. |
| Outcome: | The proposed measures show weak correlations with human evaluations and a third of the correlations show error masking. |
Personalize Your LLM: Fake it then Align it (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing personalization methods require fine-tuning of large language models for each user, rendering them prohibitively expensive for widespread adoption. |
| Approach: | They propose a retrieval-based personalization approach that uses self-generated personal preference data and representation editing to enable quick and cost-effective personalization. |
| Outcome: | The proposed approach outperforms two personalization baselines by 40% on various tasks. |
Unsupervised Paraphrasing with Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Paraphrase generation has benefited from recent advances in the design of training objectives and model architectures, but previous studies focused on supervised methods that require a large amount of labeled data that is costly to collect. |
| Approach: | They propose a transfer learning approach that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. |
| Outcome: | The proposed model performs state-of-the-art on the Quora Question Pair and ParaNMT datasets and is robust to domain shift between the two datasets. |
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial text generation approaches can lead to generation lacking diversity or fluency, whereas perturbing in the intermediate representation space can lead a model to generate generations that are not related to the input. |
| Approach: | They propose to generate adversarial texts through controllable attributes that are known to be invariant to task labels. |
| Outcome: | The proposed model generates more diverse and fluent adversarial examples, compared to existing approaches, and is more robust against model re-training and different model architectures. |
Naive Bayes-based Context Extension for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising in-context learning abilities, but are often impeded by length limitations of transformer architecture. |
| Approach: | They propose a framework to enable existing Large Language Models to perform ICL with an increased number of demonstration examples by significantly expanding their context size. |
| Outcome: | The proposed framework outperforms existing methods in in-context learning with an increased number of demonstration examples while preserving linear efficiency. |
Enhancing Recommendation Explanations through User-Centric Refinement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing explanations for user reviews often fail to meet user-centric aspects, reducing their usefulness to users. |
| Approach: | They propose a paradigm that refines initial explanations generated by existing models during the inference stage to enhance their quality in multiple aspects. |
| Outcome: | The proposed model improves explanations generated by existing models during the inference stage to enhance their quality in multiple aspects. |
Diagnosing Vision-and-Language Navigation: What Really Matters (2022.naacl-main)
Copied to clipboard
Wanrong Zhu, Yuankai Qi, Pradyumna Narayana, Kazoo Sone, Sugato Basu, Xin Wang, Qi Wu, Miguel Eckstein, William Yang Wang
| Challenge: | Existing models claim to be able to align object tokens with specific visual targets, but there are non-negligible gaps between the two. |
| Approach: | They conduct diagnostic experiments to examine how the agents perceive multimodal input by ablation diagnostics input data. |
| Outcome: | The results show that indoor and outdoor navigation agents refer to object and direction tokens when making decisions. |
INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models (2023.findings-emnlp)
Copied to clipboard
H S V N S Kowndinya Renduchintala, Krishnateja Killamsetty, Sumit Bhatia, Milan Aggarwal, Ganesh Ramakrishnan, Rishabh Iyer, Balaji Krishnamurthy
| Challenge: | Pre-trained language models have a remarkable improvement in generalization capability . however, this leads to prohibitively long training times and a detrimental environmental impact . |
| Approach: | They propose to use submodular optimization to select highly informative subsets of training data to train multiple PTLMs using only fractions of data. |
| Outcome: | The proposed framework achieves 99% of the performance of fully-trained models using only fraction of training data. |
More Diverse Dialogue Datasets via Diversity-Informed Data Collection (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate conversational dialogue produce uninteresting, predictable responses. |
| Approach: | They propose a method to collect and determine more diverse data from conversational participants . they use dynamically computed corpus-level statistics to determine which conversational participant to collect data from . |
| Outcome: | The proposed method produces significantly more diverse data than baseline methods and better results on emotion classification and dialogue generation tasks. |
MCC-KD: Multi-CoT Consistent Knowledge Distillation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable abilities in complex reasoning through chain of thought (CoT) prompting. |
| Approach: | They propose to generate multiple rationales for each question and enforce consistency among their predictions by minimizing the bidirectional KL-divergence between the answer distributions. |
| Outcome: | The proposed model achieves superior performance on in-distribution and commonsense reasoning benchmarks. |
Mitigating Negative Interference in Multilingual Knowledge Editing through Null-Space Constraints (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing monolingual knowledge editing methods are expensive and require multiple models to maintain factual consistency. |
| Approach: | They propose a null-space constrained framework to precisely isolate language-specific knowledge updates that can be mapped onto other languages’ subspaces. |
| Outcome: | The proposed framework can project parameter updates for each language onto the orthogonal complement of other languages’ subspaces while preserving multilingual generalization capabilities. |
AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets (2024.naacl-long)
Copied to clipboard
| Challenge: | Standard pool-based active learning is computationally expensive on large pools and often reaches low accuracy by overfitting the initial decision boundary. |
| Approach: | They propose a pool-based active learning method that selects class-specific instances from a labelled set and retrieves the most similar unlabelled instances from the pool. |
| Outcome: | Experiments with AnchorAL show that it is faster, often reducing runtime from hours to minutes, and trains more performant models. |
Visually Grounded Continual Language Learning with Selective Specialization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on the role of model components in learning specialized or generalizable representations are lacking. |
| Approach: | They propose to analyze selection strategies for visually grounded continual language learning using two diagnostic datasets. |
| Outcome: | The proposed models outperform existing models and provide enough control and flexibility for a thorough model analysis. |
Enhancing Structure-aware Encoder with Extremely Limited Data for Graph-based Dependency Parsing (2022.coling-1)
Copied to clipboard
| Challenge: | Dependency parsing is an important natural language processing task which analyzes the syntactic structure of an input sentence. |
| Approach: | They propose a structure-aware encoder pre-trained on auto-parsed data to improve dependency parsing . they propose combining gold dependency trees with existing parsers to improve parser performance . |
| Outcome: | The proposed approach outperforms baselines under different parsers and dependency standards under different parameters and model architectures. |
Measuring and Increasing Context Usage in Context-Aware Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has demonstrated the necessity and feasibility of using inter-sentential context, but it is often not clear how much they actually utilize it at translation time. |
| Approach: | They propose a conditional cross-mutual information metric to quantify usage of context by model architectures that can use it at translation time. |
| Outcome: | The proposed method increases context usage and improves translation quality according to BLEU and COMET metrics. |
Sharpness-Aware Minimization Improves Language Model Generalization (2022.acl-long)
Copied to clipboard
| Challenge: | Comparatively little work has been done to improve the generalization of language models . recent work shows that Sharpness-Aware Minimization (SAM) can improve generalization without much computational overhead. |
| Approach: | They propose a Sharpness-Aware Minimization procedure that encourages convergence to flatter minima to improve generalization of language models without much computational overhead. |
| Outcome: | The proposed Sharpness-Aware Minimization procedure can improve language models without much computational overhead. |
Joint Slot Filling and Intent Detection via Capsule Neural Networks (P19-1)
Copied to clipboard
| Challenge: | Existing models that label slots and detect intent do not preserve hierarchical relationship between words, slots, and intents. |
| Approach: | They propose a capsule-based neural network model which performs slot filling and intent detection via a dynamic routing-by-agreement schema. |
| Outcome: | The proposed model performs better than existing models and existing models on real-world datasets. |
A Dataset for Tracking Entities in Open Domain Procedural Text (2020.emnlp-main)
Copied to clipboard
Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson, Eduard Hovy
| Challenge: | Existing tasks require only a small set of attributes to track state changes in procedural text. |
| Approach: | They propose a task where given a procedural text as input, the task is to generate a set of state change tuples for each step. |
| Outcome: | The proposed task generates state change tuples from a set of pre-defined attributes for each step and predicts them from an open vocabulary. |
ChatMatch: Evaluating Chatbots by Autonomous Chat Tournaments (2022.acl-long)
Copied to clipboard
| Challenge: | Existing automated evaluation systems of chatbots rely on static chat scripts as ground truth, which is hard to obtain. |
| Approach: | They propose an interactive chatbot evaluation framework that allows chatbots to compete with each other like in a sports tournament. |
| Outcome: | The proposed framework can rank chatbots independently from their model architectures and domains . existing evaluation systems rely on static chat scripts as ground truth . |
Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to address hallucinations in large vision-language models require substantial computational cost and time. |
| Approach: | They propose to leverage sparse autoencoders to identify semantic directions closely associated with faithfulness or hallucination, extracting more precise and disentangled hallucinian-related representations. |
| Outcome: | The proposed method outperforms existing decoding approaches while maintaining transferability across different model architectures with negligible additional time overhead. |
Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a study shows that comprehension-intensive fine-tuning tasks retain knowledge longer . however, all models exhibit significant performance drops when applying injected knowledge in broader contexts . |
| Approach: | study: comprehension-intensive fine-tuning tasks achieve higher knowledge retention rates . larger models show improved retention across all task types, study finds . |
| Outcome: | a new study shows that comprehension-intensive fine-tuning tasks retain knowledge better than mapping-oriented tasks despite exposure to identical factual content. |
Improving Answer Selection and Answer Triggering using Hard Negatives (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to answer selection and answer triggering have been proposed. |
| Approach: | They propose to use hard negatives with a siamese network and a suitable loss function for answer selection and answer triggering. |
| Outcome: | The proposed model improves on InsuranceQA, SelQA, and an internal QA dataset by 2.3 points over previous baselines. |
SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models (2024.acl-long)
Copied to clipboard
Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, Wanxiang Che
| Challenge: | Existing methods to address catastrophic forgetting and knowledge transfer in large language models (LLMs) ignore potential of aligning the two modules to effectively address catastrophic forgetting and knowledge transfers simultaneously. |
| Approach: | They propose a Shared Attentive Learning & Selection module to align the PET learning and selection modules to address catastrophic forgetting and knowledge transfer simultaneously. |
| Outcome: | Experiments on two CL benchmarks show that the proposed framework is superior when scaled to different model sizes, different model architectures and unseen tasks. |
Quantile Regression with Large Language Models for Price Prediction (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to structured prediction tasks focus on point estimates and lack systematic comparison across different methods. |
| Approach: | They propose a novel quantile regression approach that enables LLMs to produce full predictive distributions, improving upon traditional point estimates. |
| Outcome: | The proposed model outperforms encoder architectures, embedding-based methods, and few-shot learning methods in prediction accuracy and distributional calibration. |
MAIN: Mutual Alignment Is Necessary for instruction tuning (2025.emnlp-main)
Copied to clipboard
Fanyi Yang, Jianfeng Liu, Xin Zhang, Haoyu Liu, Xixin Cao, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Instruction tuning has enabled large language models to achieve remarkable performance, yet its success heavily depends on the availability of high-quality instruction-response pairs. |
| Approach: | They propose a mutual alignment framework which enforces coherence between instructions and responses through mutual constraints. |
| Outcome: | The proposed framework generalizes well across model architectures and sizes, achieving state-of-the-art performance on LLaMA, Mistral, and Qwen models across diverse benchmarks. |
Fisher Mask Nodes for Language Model Merging (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained models are ubiquitous in natural language processing, but individual fine-tuned models require significant overhead in multi-task scenarios. |
| Approach: | They propose a method for fine-tuning pre-trained models for Transformers using Fisher information. |
| Outcome: | The proposed method outperforms Fisher-weighted averaging in a fraction of the computational cost. |
What are the Goals of Distributional Semantics? (2020.acl-main)
Copied to clipboard
| Challenge: | a new paper examines distributional semantic models' ability to deal with semantic challenges . authors argue that assessing progress in any field requires explicit long-term goals . |
| Approach: | They propose a broad linguistic perspective to assess distributional semantic models' ability to deal with various semantic challenges. |
| Outcome: | The proposed models can handle various semantic challenges, but they need to be explicit . a top-down approach is largely bottom-up, while a bottom-down one is mainly top-up . the authors argue that the goal is unclear and that the models are not scalable . |
What if Othello-Playing Language Models Could See? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a multi-modal model trained on move sequences and board images is a popular testbed for language models . |
| Approach: | They propose a multi-modal model trained jointly on move sequences and board images. |
| Outcome: | The proposed multi-modal model trains on move sequences and board images. |
AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification (2025.emnlp-main)
Copied to clipboard
Xuan Zhang, Yongliang Shen, Zhe Zheng, Linjuan Wu, Wenqi Zhang, Yuchen Yan, Qiuying Peng, Jun Wang, Weiming Lu
| Challenge: | Existing tools for ambiguous and incomplete queries are limited by manual construction and lack of error correction mechanisms during multi-turn clarification. |
| Approach: | They propose a framework that exploits the mapping between queries and their tool invocation solutions by removing key parameters from queries while retaining them as ground truth. |
| Outcome: | The proposed framework outperforms existing methods while maintaining high accuracy in tool invocation. |
DualTKB: A Dual Learning Bridge between Text and Knowledge Base (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for KB construction and sentence generation are lacking in the field of knowledge transfer. |
| Approach: | They propose a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases. |
| Outcome: | The proposed method compares favorably to existing baselines and is a viable step towards a more advanced system for automatic KB construction/expansion and reverse operation of sentence generation from KBs. |
ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning (2025.findings-emnlp)
Copied to clipboard
Yang Wu, Huayi Zhang, Yizheng Jiao, Lin Ma, Xiaozhong Liu, Jinhong Yu, Dongyu Zhang, Dezhi Yu, Wei Xu
| Challenge: | Prevailing methods for task-specific instruction tuning use similarity metrics to select training data . but instruction tuning loss often fails to exhibit a monotonic relationship with actual task performance . |
| Approach: | They propose a task-specific instruction tuning method that leverages pairwise preference loss as a reward signal. |
| Outcome: | The proposed method surpasses state-of-the-art methods for task-specific instruction tuning. |
Textual Dataset Distillation via Language Model Embedding (2024.findings-emnlp)
Copied to clipboard
| Challenge: | prevailing methods for dataset distillation generate distilled data as embedding vectors, which are not human-readable. |
| Approach: | They propose a model-agnostic, data-efficient method that leverages Language Model embeddings . their method offers enhanced flexibility and improved transferability . |
| Outcome: | The proposed method achieves comparable performance with faster processing times compared to other methods . it offers enhanced flexibility and improved transferability, expanding the range of potential applications . |
HYRR: Hybrid Infused Reranking for Passage Retrieval (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing passage retrieval systems typically adopt a two-stage retrieve-then-rerank pipeline. |
| Approach: | They propose a framework for training robust reranking models using hybrid retrievers . they propose HYRR framework that allows users to select training data using hybrids . |
| Outcome: | The proposed framework is robust to different first-stage retrieval settings. |
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue (2022.emnlp-main)
Copied to clipboard
Alon Albalak, Yi-Lin Tuan, Pegah Jandaghi, Connor Pryor, Luke Yoffe, Deepak Ramachandran, Lise Getoor, Jay Pujara, William Yang Wang
| Challenge: | Prior studies of task transfer in dialogue consider only 2-4 tasks, focus on multitasks. |
| Approach: | They propose a benchmark for FEw-sample TAsk transfer in open-domain dialogue. |
| Outcome: | The proposed benchmark analyzes the transferability between 132 source-target task pairs and provides a baseline for future work. |
Numerical reasoning in machine reading comprehension tasks: are we there yet? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Numerical reasoning based machine reading comprehension models have achieved near-human performance on a variety of benchmarks, but are they capable of learning to reason? |
| Approach: | They propose to use a DROP benchmark to measure machine reading comprehension and investigate models that have achieved near-human performance over standard metrics. |
| Outcome: | The DROP benchmark has inspired the design of specialized BERT and embedding the results into a specialized model. |
VADE: Visual Attention Guided Hallucination Detection and Elimination (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision Language Models (VLMs) are prone to hallucinations, generating outputs that lack grounding in the actual visual data. |
| Approach: | They propose a sequence modelling approach to learn complex sequential patterns from transformer attention maps. |
| Outcome: | The proposed approach achieves an average PR-AUC of 80% in hallucination detection on M-HalDetect and an 5% improvement in hallucinosis mitigation on MSCOCO. |
Improving Continual Pre-training Through Seamless Data Packing (2025.findings-acl)
Copied to clipboard
| Challenge: | Empirical evaluations across various model architectures and corpus domains demonstrate the effectiveness of our method, outperforming baselines in 99% of all settings. |
| Approach: | They propose a method that uses a sliding window technique to pack data before continual pre-training to preserve contextual information and enhance model performance. |
| Outcome: | Empirical evaluations across various model architectures and corpus domains demonstrate the effectiveness of the proposed method outperforming baselines in 99% of settings. |
SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration (2026.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models. |
| Approach: | They propose a self-draft framework that suppresses spurious confidence via layer-wise temperature annealing in early-exit decision and adaptively bounds speculation length based on token-wise decoding difficulty. |
| Outcome: | The proposed framework suppresses spurious confidence and bounds speculation length based on token-wise decoding difficulty. |
Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations (2023.acl-long)
Copied to clipboard
| Challenge: | Human-annotated labels and explanations are critical for training explainable NLP models. |
| Approach: | They propose a metric that measures the usefulness of an explanation for model performance at both fine-tuning and inference. |
| Outcome: | The proposed metric can evaluate the quality of human-annotated explanations, while Simulatability falls short. |
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? (2023.findings-emnlp)
Copied to clipboard
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Tran, Dani Yogatama, Donald Metzler
| Challenge: | Existing studies on the scaling properties of model architectures have not explored the impact of inductive biases on scaling behaviour. |
| Approach: | They conduct extensive experiments to understand scaling behaviour of ten different model architectures. |
| Outcome: | The results show that the best performing model can fluctuate at different scales. |
Digging Errors in NMT: Evaluating and Understanding Model Errors from Partial Hypothesis Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current evaluation of neural machine translation systems is limited by one best hypothesis and search errors brought by heuristic decoding algorithms. |
| Approach: | They propose a new evaluation protocol which defines model errors with model’s ranking capability over hypothesis space and Monte Carlo sampling evaluation to tackle the problem of exponentially large space. |
| Outcome: | The proposed evaluation protocol is consistent with what is currently used in the field and is consistent to what is being proposed. |
Identifying Noise in Human-Created Datasets using Training Dynamics from Generative Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing noise detection techniques for autoencoder models do not generalize to ArLMs due to differences in learning dynamics. |
| Approach: | They propose a method that leverages training dynamics to rank datapoints from easy-to-learn to hard-tolear . TDRanker achieves at least 2x faster denoising than previous techniques . |
| Outcome: | The proposed method demonstrates robustness across multiple model architectures and noise levels. |
Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Arabic diacritics are typically omitted in written Arabic, leading to ambiguity . authors propose a methodology to analyze and refine a large diacritized corpus . |
| Approach: | They propose a methodology to analyze and refine a large diacritized corpus to improve training quality. |
| Outcome: | The proposed model achieves state-of-the-art results with 3.12% and 2.70% WER on WikiNews-2014 and Wikinews-2024. |
Triviality Corrected Endogenous Reward (2026.acl-long)
Copied to clipboard
Xinda Wang, Zhengxu Hou, Yangshijie Zhang, null Yanbingren, Jialin Liu, ChenZhuo Zhao, Zhibo Yang, Bin-Bin Yang, Feng Xiao
| Challenge: | Recent work on unsupervised reinforcement learning for mathematical reasoning using confidence-based endogenous rewards focuses on open-ended text generation, requiring either annotated data or powerful closed-source models. |
| Approach: | They propose a method that rewards the relative information gain between a specialist and a generalist reference policy, modulated by a probability-dependent correction mechanism. |
| Outcome: | The proposed model improves on multiple writing benchmarks and model architectures without external supervision and validates generality across different generation tasks. |
Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to decode text to the most probable sequence have been proposed to address these challenges by improving coherence, diversity, and resemblance to human-generated text. |
| Approach: | They propose a novel decoding strategy that extends contrastive search by incorporating an adaptive degeneration penalty informed by the model’s estimated uncertainty at each generation step. |
| Outcome: | The proposed approach improves creativity and coherence while maintaining coherency across model architectures, languages, and datasets. |
A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) display impressive performances and have captured the attention of the NLP community. |
| Approach: | They propose to compare multilingual pretraining objectives in a controlled methodological environment with multilingual models. |
| Outcome: | The proposed model outperforms existing models in 6 languages and demonstrates that multilingual translation is an effective pretraining objective under the right conditions. |
Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism (2025.findings-emnlp)
Copied to clipboard
Zhiwei Wang, Yunji Wang, Zhongwang Zhang, Zhangchen Zhou, Hui Jin, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Yaoyu Zhang, Zhi-Qin John Xu
| Challenge: | Large language models struggle with complex reasoning tasks, such as mathematical problem-solving. |
| Approach: | They constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning. |
| Outcome: | The proposed algorithm improves on 7 multi-step reasoning datasets, while introducing only 132 trainable parameters. |
Challenges in Context-Aware Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | despite well-reasoned intuitions, most context-aware neural machine translation models show only modest improvements over sentence-level systems. |
| Approach: | They propose a more realistic setting for document-level translation called paragraph-to-paragraph (PARA2PARA) they collect a dataset of Chinese-English novels to promote future research . |
| Outcome: | The proposed model improves translation quality across document-level metrics and discourse phenomena. |
Rethinking Word-Level Auto-Completion in Computer-Aided Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for word-level auto-completion (WLAC) do not meet the criterion of good auto-completes. |
| Approach: | They propose a measurable criterion to address the question: what kind of words are good auto-completions? they propose an approach to enhance WLAC performance by promoting adherence to the cri-terion. |
| Outcome: | The proposed approach outperforms the top-performing system submitted to the WLAC shared tasks in WMT2022 while using significantly smaller model sizes. |
Systematic Evaluation of Auto-Encoding and Large Language Model Representations for Capturing Author States and Traits (2025.findings-acl)
Copied to clipboard
Khushboo Singh, Vasudha Varadarajan, Adithya V Ganesan, August Håkan Nilsson, Nikita Soni, Syeda Mahwish, Pranav Chitale, Ryan L. Boyd, Lyle Ungar, Richard N Rosenthal, H. Schwartz
| Challenge: | Large Language Models (LLMs) are increasingly used in human-centered applications, yet their ability to model diverse psychological constructs is not well understood. |
| Approach: | They evaluated a range of Transformer-LMs to predict psychological variables across five major dimensions: affect, substance use, mental health, sociodemographics, and personality. |
| Outcome: | The models predict affect, substance use, mental health, sociodemographics, and personality across five major dimensions. |
Looking Beyond Text: Reducing Language Bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) have been criticized for their language bias. |
| Approach: | They propose to use a dual-attention mechanism to construct separate attention for visual and text inputs to enhance integration of visual inputs across models. |
| Outcome: | Experiments show that the proposed model debiases LVLMs from their language bias, enhancing visual comprehension and reducing hallucinations without additional resources. |
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have shown remarkable capabilities, particularly in English, but for less prevalent languages, performance can be significantly lower, making additional adaptation paramount. |
| Approach: | They propose a new adaptation method based on iteratively merging multiple models fine-tuned on a subset of available training data that reduces forgetting while maintaining learning on the target domain. |
| Outcome: | The proposed method outperforms LLAMA-3-8B-based models in German and German while maintaining learning on the target domain. |
From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current research on bias in language models focuses on data quality, not temporal influences of data. |
| Approach: | They propose a methodology to interpret the interaction between training data and model architecture in bias propagation during language modeling. |
| Outcome: | The proposed method analyzes the interaction between training data and model architecture in bias propagation during language modeling. |
Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction (2025.acl-long)
Copied to clipboard
| Challenge: | Unlike traditional dialogue systems, goal-oriented proactive dialogue systems focus on achieving specific objectives by actively guiding and anticipating user needs. |
| Approach: | They propose a model-agnostic two-stage Consistency Reflection and Correction framework that allows the model to reflect on discrepancies between generated responses and dialogue contexts and suggest possible corrections. |
| Outcome: | The proposed framework significantly improves the consistency between generated responses and dialogue contexts on three datasets. |
Query-driven Relevant Paragraph Extraction from Legal Judgments (2024.lrec-main)
Copied to clipboard
| Challenge: | Legal professionals struggle with navigating lengthy legal judgements to pinpoint information that directly addresses their queries. |
| Approach: | They construct a specialized dataset to extract relevant paragraphs from legal judgements based on query . they assess the performance of current retrieval models in a zero-shot way . |
| Outcome: | The proposed model outperforms the current retrieval models in a zero-shot way and fine-tunes them using various models. |
Rethinking Text-based Protein Understanding: Retrieval or LLM? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment. |
| Approach: | They propose a retrieval-enhanced method which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. |
| Outcome: | The proposed method significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. |
OptiCo: Adaptive Distributed Training Optimization via Collaborative Agent Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing distributed training frameworks are plagued by over-reliance on prior profiling and poor generalization across models/hardware. |
| Approach: | They propose a model-driven multi-agent framework that leverages Large Language Models to enable automatic and explainable distributed training strategy configuration. |
| Outcome: | The proposed framework outperforms expert-designed training strategies within 20 iterations. |
Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains (2025.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language models struggle to disentangle information scattered across complex visual inputs, leading to performance degradation. |
| Approach: | They propose a focus-centric visual chain paradigm that enhances VLMs’ perception, comprehension, and reasoning abilities in multi-image scenarios. |
| Outcome: | The proposed approach achieves average performance gains of 3.16% and 2.24% across two distinct model architectures, without compromising the general vision-language capabilities. |
Online Iterative Self-Alignment for Radiology Report Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for RRG rely on supervised fine-tuning based on data pairs of radiological images and corresponding radiologist-annotated reports. |
| Approach: | They propose a method that performs supervised fine-tuning on data pairs of radiological images and corresponding radiologist-annotated reports. |
| Outcome: | The proposed method surpasses existing methods and achieves state-of-the-art performance across multiple evaluation metrics. |
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks (2026.findings-acl)
Copied to clipboard
| Challenge: | Survey aims to identify challenges of multimodal unlearning for vision, language, audio and video . retraining after deletion requests or policy updates is often impractical, survey finds . |
| Approach: | They propose to enable selective removal across modalities while retaining overall utility. |
| Outcome: | This study compares models with existing models to identify weaknesses and improves performance. |
Structure-aware Domain Knowledge Injection for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Structure-aware Continual Pre-Training (SCPT) and Structure-Aware Supervised Fine-Tuning (SSFT) are two-stage strategies for knowledge injection and alignment that reduces the training corpus needs to 5% while achieving 100% of traditional knowledge injection performance. |
| Approach: | They propose a method to efficiently transform foundation Large Language Models into domain specialists by using two-stage strategies: Structure-aware Continual Pre-Training and Structure-Aware Supervised Fine-Tuning. |
| Outcome: | The proposed method significantly reduces the training corpus needs to a mere 5% while achieving 100% of traditional knowledge injection performance. |
DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents (2026.findings-acl)
Copied to clipboard
Kai Shi, Jun Yang, Ni Yang, Binqiang Pan, Qingsong Xie, null Zhangchao, Zhenyu Yang, Tianhuang Su, Haonan Lu
| Challenge: | Mobile Phone Agents (MPAs) have attracted huge attention due to their practicability in a multitude of scenarios. |
| Approach: | They propose a data mixture optimization solution that extrapolates optimal data mixtures from a trainable network. |
| Outcome: | The proposed model outperforms existing methods on open-source benchmarks and on open source benchmarks. |
A Survey of Inductive Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Kedi Chen, Dezhao Ruan, Yuhao Dan, Yaoting Wang, Siyu Yan, Xuecheng Wu, Yinqi Zhang, Qin Chen, Jie Zhou, Liang He, Biqing Qi, Linyang Li, Qipeng Guo, Xiaoming Shi, Wei Zhang
| Challenge: | Inductive reasoning is an important task for large language models (LLMs). |
| Approach: | They propose a survey of inductive reasoning for large language models . they categorize methods into three main areas: post-training enhancement, test-time exploration, and data augmentation. |
| Outcome: | The proposed method improves inductive reasoning in large language models. |
LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark (2026.findings-acl)
Copied to clipboard
Guangyi Liu, Pengxiang Zhao, Liang Liu, Zhiming Chen, Yuxiang Chai, Yaozhen Liang, WenHao Wang, Siheng Chen, Zhengxi Lu, Shuai Ren, Hao Wang, Shibo He, Yong Liu, Wenchao Meng
| Challenge: | Mobile GUI agents show promise in automating tasks but face significant generalization challenges in long-tail scenarios. |
| Approach: | They propose a benchmark framework for mobile GUI agents that measures the performance of GUI agents by analyzing their performance. |
| Outcome: | The LearnGUI benchmark outperforms existing methods in offline and online evaluations and demonstrates consistent gains across model architectures. |
Behavior Knowledge Merge in Reinforced Agentic Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for supervised fine-tuning (SFT) are suboptimal to preserve task-specific capabilities on RL-trained agentic models. |
| Approach: | They propose a distribution-aware merging framework specifically designed for RL-trained agentic models that disentangles shared and task-specific unique parameter updates while selectively preserving and rescaling unique ones. |
| Outcome: | Experiments across multiple agent domains and model architectures show that the proposed framework surpasses baselines and unlocks synergistic potential among agents. |
LoPT: Lossless Parallel Tokenization Acceleration for Long Context Inference of Large Language Model (2026.acl-long)
Copied to clipboard
| Challenge: | Existing parallel tokenization methods suffer from inconsistent results due to boundary artifacts that occur after merging. |
| Approach: | They propose a Lossless Parallel Tokenization framework that ensures output identical to standard sequential tokenization. |
| Outcome: | The proposed method achieves significant speedup while guaranteeing lossless tokenization. |
PseudoGD: Enhancing Spatial Reasoning in Vision-Language Models through Pseudo Geometric Knowledge Distillation (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent Large Vision-Language Models (LVLMs) have shown remarkable success in general semantic understanding, but struggle with 3D spatial reasoning tasks. |
| Approach: | They propose a framework to help vision encoders internalize 3D geometric information using only standard 2D images. |
| Outcome: | The proposed framework achieves State-of-the-Art (SOTA) performance across various model architectures. |
Can LLMs Extract Frame-Semantic Arguments? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Frame-semantic parsing is a critical task in natural language understanding . however, the ability of large language models to extract frame-sensical arguments remains unexplored . |
| Approach: | They propose a framework to extract frame-semantic arguments from large language models . they use JSON representations to enhance performance, but smaller models can achieve competitive results . |
| Outcome: | The proposed model achieves state-of-the-art on ambiguous targets while limiting generalization to out-of domain data. |
Towards Intrinsic Interpretability of Large Language Models: A Survey of Design Principles and Architectures (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on explainable AI focus on post-hoc explanation methods that interpret trained models through external approximations. |
| Approach: | They propose to categorize existing approaches into five design paradigms: functional transparency, concept alignment, representational decomposability, explicit modularization, and latent sparsity induction. |
| Outcome: | The proposed approaches are categorized into five design paradigms: functional transparency, concept alignment, representational decomposability, explicit modularization, and latent sparsity induction. |
TTVS: Boosting Self-Exploring Reinforcement Learning via Test-time Variational Synthesis (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing test-time methods are limited in specialized or novel domains where supervision is prohibitively expensive or unavailable. |
| Approach: | They propose a framework that augments training stream from unlabeled test queries. |
| Outcome: | Extensive experiments show TTVS outperforms state-of-the-art RL-based techniques on unlabeled test-time data. |
Towards Unified Multimodal Large Language Models: A survey (2026.findings-acl)
Copied to clipboard
| Challenge: | unified multimodal large language models (MLLMs) are emerging but lack a systematic framework to connect them and situate current trends within a broader landscape. |
| Approach: | They present a systematic review of unified Multimodal Large Language Models . they outline the foundational concepts and prerequisites for understanding them . |
| Outcome: | The present review provides a systematic and systematic overview of unified MLLMs . it discusses persistent challenges and identify promising directions for future research . |
Autoregressive Semantic Visual Reconstruction Helps VLMs Understand Better (2026.findings-acl)
Copied to clipboard
| Challenge: | Typical large vision-language models emphasize vision-to-language alignment while overlooking fine-grained visual information. |
| Approach: | They introduce autoregressive semantic visual reconstruction (ASVR) that enables joint learning of visual and textual modalities within a unified autoregression framework. |
| Outcome: | The proposed model improves baselines and multimodal understanding benchmarks by 2-3%. |
AdaFuse: Adaptive Ensemble Decoding for Large Language Models (2026.acl-long)
Copied to clipboard
Chengming Cui, Tianxin Wei, Ziyi Chen, Ruizhong Qiu, Zhichen Zeng, Zhining Liu, Xuying Ning, Duo Zhou, Jingrui He
| Challenge: | Existing ensemble approaches to large language models lack flexibility for mid-generation adaptation. |
| Approach: | They propose an adaptive ensemble decoding framework that dynamically selects semantically appropriate fusion units during generation. |
| Outcome: | The proposed framework outperforms existing ensemble frameworks on open-domain QA, arithmetic reasoning, and machine translation tasks. |
Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026.acl-long)
Copied to clipboard
Tingyu Song, Yanzhao Zhang, Mingxin Li, Zhuoning Guo, Dingkun Long, Pengjun Xie, Siyue Zhang, Yilun Zhao, Shu Wu
| Challenge: | Composed Image Retrieval (CIR) is a complex task in multimodal understanding . current CIR benchmarks lack a robust evaluation pipeline and limited query categories . |
| Approach: | They construct a fine-grained CIR benchmark that allows for precise control over modification types and content. |
| Outcome: | The proposed benchmark covers 5,000 high-quality queries structured across five main categories and fifteen subcategories. |
CHOIR: Harmonizing Structured Persona Diversity for Robust Collaborative LLM Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Persona-assigned Large Language Models can be useful for personalized, context-aware reasoning. |
| Approach: | They propose a framework that harmonizes demographic perturbations into a unified prediction by balancing agreement and divergence among counterfactual personas. |
| Outcome: | The proposed framework improves reasoning performance even when base personas are suboptimal. |